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FOREWORD 


The  Fort  Leavenworth  Field  Unit  of  the  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  (ARI)  supports  the  Combined  Arms  Center  with 
research  and  development  on  combined  arms  operations  and  command  group  train¬ 
ing.  With  the  fielding  of  the  first  fully  automated  systems  for  command  a no 
staff  training,  the  simulation  development  community  is  engaged  in  assessing 
lessons  from  previous  efforts  while  embarking  on  the  automation  of  staff 
training  at  high  echelons. 

As  part  of  Research  Task  1.3.3.,  "Improved  Methods  for  Command  Group 
Training,"  the  report  summarizes  10  years  of  research  on  issues  from  which  de¬ 
sign  criteria  may  be  derived  for  the  new  generation  of  training  systems.  The 
report  was  prepared  under  a  Memorandum  of  Understanding,  dated  30  May  1983, 
between  the  Combined  Arms  Training  Activity  (CATA)  and  the  Army  Research  In¬ 
stitute.  Recommendations  from  the  report,  which  were  briefed  to  the  Training 
Simulations  System  Manager  of  CATA  on  30  January  1987,  will  be  incorporated  in 
requirements  for  new  simulations. 


/ 


EDGAR  M.  JOHNSON 
Technical  Director 
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DESIGN  OF  BATTLE  SIMULATIONS  FOR  COMMAND  AND  STAFF  TRAINING 


EXECUTIVE  SUMMARY 


Requirement : 

To  provide  design  criteria  for  the  next  generation  of  automated  battle 
simulation  based  on  behavioral  research  on  command  and  staff  training. 


Procedure : 

A  systems-approach-to-training  model  was  adapted  to  the  context  of  com¬ 
mand  and  staff  training  with  automated  battle  simulations.  Results  of  re¬ 
search  on  each  of  the  major  components  of  the  model  were  discussed  and  related 
to  design  goals  and  criteria  that  should  be  considered  in  the  development  of 
future  training  systems. 


Findings : 

Excessive  support  staff  requirements,  lack  of  system  support  for  scenario 
development,  lack  of  system  control  over  information  and  intelligence  and  lack 
of  performance  measurement  capabilities  were  found  to  be  recurrent  deficien¬ 
cies  in  previously  developed  systems.  A  new  requirement  for  training  with 
tactical  data  systems  may  ameliorate  some  of  these  deficiencies  while  adding 
to  the  complexity  of  system  software.  Proposed  solutions  include  developing  a 
systemic  model  to  minimize  support  requirements  and  using  on-line  data  captur¬ 
ing  techniques  for  performance  measurement. 


Utilization  of  Findings: 

Application  of  these  lessons  learned  will  allow  developers  of  training 
systems  to  avoid  previous  problems  in  user  acceptance  of  systems.  Incorpora¬ 
tion  of  the  recommendations  in  the  new  generation  of  training  systems  will 
reduce  support  costs  and  increase  the  ability  of  such  systems  to  assess  and 
address  training  needs. 
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DESIGN  OF  BATTLE  SIMULATIONS  FOR  COMMAND  AND  STAFF  TRAINING 


SECTION  1.  INTRODUCTION 

Simulations  designed  to  train  Army  commanders  and  staffs  in  planning  and 
conducting  military  operations  have  reached  a  turning  point  in  their  develop¬ 
ment.  Manual  simulations  at  lower  echelons  (battalion  and  brigade)  have  been 
supplemented  and  to  some  extent  supplanted  by  computer-assisted  simulations. 
Currently,  the  Army  is  fielding  the  first  operational  f ully-automated  simula¬ 
tions.  The  availability  of  low-cost  high-performance  computer  systems  and 
high-fidelity  combat  models  has  made  it  possible  to  design  the  next  generation 
of  training  devices  based  upon  the  desired  performance  capabilities  of  the 
system  rather  than  being  overly  constrained  by  limitations  in  technology.  The 
simulation  development  community  is  now  engaged  in  assessing  the  lessons 
learned  in  previous  development  efforts  while  embarking  upon  a  much  more  ambi¬ 
tious  plan  to  automate  staff  training  at  high  echelons. 

The  Army  Research  Institute  (ARI)  Field  Unit  at  Fort  Leavenworth  has  been 
participating  in  battle  simulation  research  and  development  for  ten  years, 
including  work  on  CAMMS,  CATTS ,  ARTBASS,  and  MACE.  During  that  decade,  an 
accumulation  of  experience  and  research  findings  has  developed  that  can  be 
brought  to  bear  upon  the  design  of  the  next  generation  of  simulations.  This 
report  documents  some  of  the  major  training  issues  and  design  goals  that  should 
be  considered  in  battle  simulation  development. 


Background 

The  basic  features  of  a  systems  approach  to  training  are  depicted  in  Figure 
1.  The  training  objectives  are  comprised  of  a  list  of  tasks  that  the  trainees 
should  perform,  a  list  of  conditions  under  which  they  should  be  performed,  and 
a  set  of  standards  for  performance.  In  the  case  of  battle  simulations,  the 
training  system  consists  of  a  simulated  battlefield  situation  that  should  be 
designed  to  require  performance  of  the  tasks  included  in  the  training  objec¬ 
tives  (or  a  subset  of  those  tasks)  while  simulating  the  conditions  under  which 
those  tasks  would  be  required.  Figure  1  emphasizes  the  role  of  feedback  in  the 
conduct  of  training  and  in  determining  which  objectives  require  additional 
training. 

Feedback  is  critical  for  effective  learning.  A  major  reason  to  adopt  a 
simulation-based  training  approach  is  to  increase  the  amount  of  task  feedback 
obtained  by  the  trainees.  Command  groups  receive  this  feedback  from  the  simu¬ 
lation,  from  subordinates,  from  superiors,  and  from  other  members  of  the  staff. 


The  Combined  Arms  Map  Maneuver  System,  the  Combined  Arms  Tactical  Training 
Simulation,  and  the  Army  Training  Battle  Simulation  System,  respectively.  MACE 
(not  an  acronym)  has  been  renamed  BABAS  -  Battalion  Battle  Simulation. 
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MEASUREMENT  &  FEEDBACK 


Unfortunately,  the  feedback,  received  is  not  as  immediate  nor  as  accurate  as  in 
simulations  designed  to  train  simple  judgmental  and  motor  skills,  such  as 
flight  simulations.  Instead,  the  root  causes  of  problems  are  often  unknown. 

In  addition,  inappropriate  decisions  and  actions  may  sometimes  lead  to  success, 
while  appropriate  behavior  may  occasionally  lead  to  failure.  For  these  rea¬ 
sons,  the  intrinsic  feedback  must  be  supplemented  with  an  extrinsic  assessment 
of  performance.  Furthermore,  this  measurement  requirement  may  require  modifica¬ 
tions  in  the  design  of  the  training  system. 

Figure  1  shows  how  the  general  training  model  is  typically  applied  to  com¬ 
mand  staff  training.  A  set  of  training  objectives,  based  upon  a  prior  assess¬ 
ment  of  the  training  needs  of  a  command  group,  is  used  to  select  and  embellish 
a  scenario  which  will  require  the  staff  to  perform  the  tasks  in  the  set.  The 
training  objectives  are  also  used  to  develop  a  plan  for  observation  and  meas¬ 
urement  of  staff  performance.  The  scenario  consists  of  a  mission,  resources 
sufficient  to  accomplish  the  mission,  an  opponent,  a  recent  history  of  the  con¬ 
flict,  a  piece  of  terrain,  and  an  intelligence  build-up  giving  clues  to  the 
size,  location,  and  probable  course  of  action  of  the  opponent.  The  basic  fea¬ 
tures  of  the  scenario  are  used  by  the  control  personnel  to  prepare  an  opera¬ 
tions  order  for  delivery  to  the  training  audience  (the  commander  and  staff 
being  trained),  to  establish  initial  conditions  in  the  simulation  model  and,  in 
units  equipped  with  tactical  data  systems,  to  initialize  their  data  bases  as 
well. 

During  an  exercise,  the  control  personnel  simulate  higher,  lower,  and  adja¬ 
cent  headquarters;  they  translate  command  group  directives  into  computer  input 
formats;  they  translate  computer  output  into  tactical  messages;  they  keep  them¬ 
selves  apprised  of  the  tactical  situation  and  of  the  status  of  resources  in 
their  functional  area;  they  supply  additional  model  control  input  for  the  op¬ 
posing  force;  and  they  participate  in  observation  and  measurement  of  staff 
performance . 

The  commander  and  staff  of  the  group  being  trained  analyze  their  mission, 
develop  a  plan,  deliver  an  operations  order,  and  execute  their  plan.  With  the 
advent  of  tactical  data  systems,  the  staff  consults  the  system  for  information, 
files  information  from  subordinate  elements,  and  provides  standard  operating 
procedure  (SOP)  reports  via  digital  transmission.  The  control  personnel  can 
also  file  SOP  reports  using  the  tactical  data  systems,  and,  if  properly  equip¬ 
ped,  can  access  the  information  residing  there  to  determine  what  picture  of  the 
battle  is  currently  available  to  the  members  of  the  training  audience. 

Organization  of  Report 

Figure  2  shows  some  of  the  steps  involved  in  developing  a  simulation-based 
training  system.  The  organization  of  this  report  is  aligned  with  the  steps  in 
the  figure.  Although  other  characterizations  are  possible,  the  steps  in¬ 
dicated  affect  the  most  basic  training  decisions  -  who  is  trained,  what  they 
are  trained  to  do,  how  their  progress  is  measured  -  and  the  most  basic  design 
decisions  -  what  modeling  techniques  are  used,  how  much  automated  support  is 
provided,  how  large  a  support  staff  is  required,  and  what  hardware  capabilities 
are  needed  to  accommodate  the  model  and  the  users  of  the  system. 
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Figure  2.  Steps  in  designing  a  battle  simulation 


The  determination  of  who  should  be  trained  in  a  battle  simulation  and  of 
what  they  should  be  trained  to  do  is  discussed  in  Section  2  of  this  report. 

The  commonplace  assumption  that  every  member  of  the  staff  group  should  be 
trained  simultaneously  on  all  of  their  combat  tasks  is  critically  examined. 

The  influence  of  performance  assessment  techniques  upon  system  design  is 
emphasized  in  Section  3.  Performance  measurement  has  been  neglected  in  exist¬ 
ing  simulations.  Usually,  the  more  promising  approaches  to  objective,  auto¬ 
mated  measurement  and  feedback  cannot  be  retroactively  incorporated. 

Simulation  design  and  fidelity  issues  are  considered  in  Section  4.  A  com¬ 
prehensive  analysis  of  modeling,  system  usage,  and  support  requirements  is 
recommended . 

The  determination  of  control  personnel  tasks  and  estimation  of  controller 
workload  are  examined  in  Sections  5  and  6,  respectively.  The  design  process  is 
viewed  as  iterative,  with  the  key  constraints  being  upon  the  availability  and 
qualifications  of  control  personnel. 
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SECTION  2.  TRAINING  OBJECTIVES 


The  basic  goal  of  simulation-based  training  is  to  allow  members  of  the 
command  group  to  acquire  the  skills  necessary  to  operate  smoothly  within  the 
time  and  resource  constraints  they  should  expect  in  battle.  Ideally,  the  con¬ 
duct  of  each  training  exercise  is  guided  by  explicit  training  objectives  which 
specify  who  is  to  be  trained,  what  tasks  they  are  being  trained  to  perform, 
under  wnat  conditions  the  tasks  must  be  performed,  and  what  criteria  or  stan¬ 
dards  indicate  successful  task  performance.  The  development  of  simulation 
systems  for  command  and  staff  training  must  be  guided  by  an  analysis  of  the 
range  of  training  objectives  for  which  the  system  might  be  employed. 

The  systems-approach-to-training  (SAT)  focuses  on  the  critical  tasks  that 
must  be  performed  to  achieve  minimal  proficiency.  SAT  is  sometimes  criticized 
as  being  an  incomplete  approach  to  training  because  of  this  focus:  it  does 
not  emphasize  the  factors  that  distinguish  true  expertise  from  acceptable  com¬ 
petence.  For  any  given  group  of  people  who  achieve  acceptable  task  performance 
in  a  SAT-based  training  system,  there  will  be  a  few  individuals  who  are  consid¬ 
ered  outstanding  at  their  job,  a  larger  group  who  are  considered  average,  and 
some  who  are  marginal.  It  has  been  contended  that  close  study  of  how  excep¬ 
tional  performers  do  things  differently  in  command  and  control  may  lead  to  the 
identification  of  a  teachable  set  of  battle  management  skills  that  could  pro¬ 
vide  much  richer  training  objectives.  Command  and  control  researchers  are 
beginning  to  investigate  these  issues.  It  remains  to  be  seen  whether  the  re¬ 
sults  could  best  be  applied  as  additional  selection  criteria,  as  improved 
training  techniques,  or  as  decision  aids.  In  the  near  term  the  SAT  approach  is 
the  only  viable  alternative. 

In  this  section,  several  implications  of  the  analysis  of  training  objec¬ 
tives  are  addressed.  First,  the  analyses  produced  by  the  Army  Training  and 
Evaluation  Program  (ARTEP)  are  discussed,  and  suggestions  are  made  for  develop¬ 
ing  additional  detail  about  objectives.  Next,  some  considerations  are  pre¬ 
sented  about  the  definition  of  the  training  audience.  Finally,  the  view  is 
presented  that  training  objectives  should  be  considered  during  scenario  devel¬ 
opment,  and  that  scenario  specification  should  be  facilitated  by  the  simulation 
system. 

ARTEP:  Tasks,  Conditions,  and  Standards 

The  ARTEP  manuals  contain  lists  of  training  objectives  which  include  tasks 
to  be  performed,  conditions  under  which  the  tasks  are  performed,  and  standards 
which  should  be  achieved.  The  lists  have  been  developed  for  commanders  and 
staffs  at  various  echelons  and  for  various  unit  types. 

The  tasks  specified  by  the  command-staff  ARTEP  manuals  focus  on  the  prod¬ 
ucts  that  should  be  produced  by  various  staff  elements  (e.g.,  estimates,  plans 
orders,  reports)  and  to  a  lesser  extent  upon  the  procedures  they  should  follow 
in  producing  the  products.  Often,  the  ARTEP  manuals  list  tasks  that  the  com¬ 
mand  group  must  accomplish,  without  indicating  which  staff  section  or  which 
individual  should  carry  out  the  tasks. 


The  conditions  listed  for  performance  of  ARTEP  tasks  focus  almost  exclu¬ 
sively  on  missions  and  tactical  situations.  Several  aspects  of  command  group 
training  which  a  simulation  should  be  designed  to  accommodate  are  neglected.  A 
few  examples  include  training  to  deal  with  incomplete,  unreliable,  contradic¬ 
tory,  and  incorrect  information  and  intelligence;  to  operate  under  time  pres¬ 
sure;  to  conduct  extended  operations  requiring  the  use  of  separate  shifts;  to 
continue  operations  with  the  loss  of  one  or  more  key  staff  members;  to  continue 
operations  with  the  loss  of  a  tactical  data  system;  to  recognize  and  deal  with 
deception;  and  to  make  decisions  under  conditions  of  fatigue,  noise  and  discom¬ 
fort.  Some  obvious  implications  of  this  partial  listing  of  additional  condi¬ 
tions  are  that  the  training  system  should  be  designed  to  operate  for  extended 
periods  of  time,  and  that  it  should  provide  for  filtering  and  modification  of 
ground  truth  information  from  the  simulation.  From  these  examples,  it  should  be 
clear  that  an  expanded  view  of  training  conditions  can  provide  useful  design 
criteria.  Conditions  will  be  discussed  further  in  sections  4  and  5. 

The  standards  for  performance  comprise  the  third  aspect  of  training  objec¬ 
tives.  ARTEP  standards  for  command  and  staff  functions  tend  to  be  vague  and 
somewhat  general  (e.g.,  accurate,  timely,  responsive,  etc.).  In  order  to  pro¬ 
vide  for  measurement  of  progress  and  assessment  of  training  needs,  these  stan¬ 
dards  must  be  supplemented.  This  issue  will  be  considered  further  in  Section 
3. 

Despite  their  limitations,  the  command  staff  ARTEP  manuals  are  the  best 
available  source  of  information  on  what  the  training  objectives  of  command 
group  training  should  be,  although  they  include  some  objectives  which  are 
better  suited  to  training  in  field  exercises  than  in  command  post  exercises. 

To  be  of  use  to  simulation  training  system  developers,  the  ARTEP  task 
lists  must  be  screened  for  inappropriate  objectives.  They  also  must  be  ex¬ 
panded  to  include  detailed  descriptions  of  the  procedures  involved  in  task 
accomplishment  and  an  assignment  of  individual  and  group  responsibility.  It 
would  be  useful  to  trace  the  interrelationships  among  tasks  and  subtasks,  as 
this  information  will  facilitate  the  subsequent  development  of  measurement 
procedures.  The  actual  conditions  under  which  the  task  might  have  to  be  per¬ 
formed  still  need  to  be  specified  because  the  ARTEP  analysis  is  limited  to 
general  situations,  i.e.,  missions,  in  which  the  tasks  are  performed.  Finally, 
the  standards  must  be  made  more  precise  and  measurable. 

Training  Audience 

A  typical  error  which  occurs  when  determining  who  must  be  in  the  training 
audience  is  to  include  too  many  people.  Attempting  to  train  the  entire  command 
group  tends  to  include  personnel  in  the  training  audience  simply  to  simulate 
the  performance  of  minor  functions  that  are  rarely  required. 

In  research  involving  12  battalion  command  staffs  (Kaplan,  1984),  ARTBASS 
participants  were  asked  to  select  training  objectives  from  a  list  of  52  poten¬ 
tial  objectives  and,  after  the  exercise,  to  rate  how  much  they  had  learned 
about  each  of  the  objectives  selected.  ARTBASS  proved  to  be  an  effective 
trainer  for  the  S3,  commander,  S2  and  FSO,  but  trained  only  half  of  the  objec¬ 
tives  selected  by  the  S4,  while  the  SI  reported  that  he  learned  little  about 
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any  of  his  objectives.  The  relative  inequality  of  training  across  staff  posi¬ 
tions  is  by  no  means  limited  to  ARTBASS  but  is  a  general  feature  of  battlefield 
simulations.  Partly  the  problem  is  the  result  of  exercises  which  begin  with  a 
scenario  in  which  the  forces  are  not  yet  engaged  and  the  units  are  fully  manned 
and  equipped.  Those  in  combat  and  combat  support  functions  can  begin  work  im¬ 
mediately  while  those  in  combat  service  support  (CSS)  functions  have  little  to 
do  during  the  initial  period.  Even  beyond  this  source  of  inequality,  however, 
the  simulation  systems  usually  provide  less  challenging  tasks,  less  varied  ac¬ 
tivity,  and  fewer  problems  for  the  CSS  sections. 

Previous  attempts  to  deal  with  this  inequality  of  training  opportunity 
include  the  development  of  off-line  manual  simulations  to  be  played  in  conjunc¬ 
tion  with  the  automated  system,  e.g.,  ADMINMOD  and  LOGMOD  for  CAMMS,  the  use  of 
probe  messages  to  stimulate  activity  in  administrative  and  logistics  areas,  and 
the  development  of  medical  and  maintenance  simulations  in  BABAS.  In  the  case 
of  the  FSO,  expansion  of  the  training  audience  to  include  fire  direction  center 
personnel  dramatically  improved  the  realism  of  his  training. 

When  determining  the  training  audience,  the  quality  of  the  activity  the 
players  perform  must  be  considered.  Simply  because  a  player  is  kept  active 
during  the  exercise  does  not  guarantee  a  training  benefit.  A  division  chaplain 
may  receive  messages  about  soldiers  who  need  counseling  or  a  veterinarian  about 
meat  which  may  be  contaminated.  This  will  give  them  something  to  report  at  the 
end  of  the  day  but  does  not  provide  them  training  in  how  to  counsel  soldiers  or 
inspect  meat.  In  contrast,  the  G3,  during  the  exercise,  is  able  to  make  plans 
and  practice  his  job.  When  deciding  who  will  be  included  in  the  training 
audience,  particularly  in  high  echelon  simulation  systems,  it  should  be  deter¬ 


mined  whether  the  person  will  perform  his  function  in  a  simulated  exercise  or 


will  merely  simulate  performance  of  his  function. 

Scenario  Development 

The  scenario  used  in  command  group  training  is  the  principal  means  of  con¬ 
trol  over  the  exercise.  The  basic  situation,  the  activity  of  the  opposing 
force,  the  intelligence  available  and  the  resources  under  the  control  of  the 
training  audience  must  be  carefully  selected  to  insure  that  all  training  objec¬ 
tives  selected  for  a  particular  exercise  are  addressed.  Ideally,  a  battle 
simulation  system  should  be  flexible  enough  to  exercise  any  scenario  consistent 
with  the  training  objectives.  In  practice,  current  automated  systems  are  lim¬ 
ited  by  the  available  graphics  and  terrain  modeling  data  (either  videodisk 
images  or  Defense  Mapping  Agency  data),  and  further  limited  in  terms  of  the 
types  of  opposing  forces  for  which  data  exists.  Although  a  small  set  of  stan¬ 
dardized  scenarios  is  usually  developed  in  conjunction  with  a  battle  simula¬ 
tion,  primarily  for  operational  testing  purposes,  existing  simulation  systems 
provide  very  limited  support  for  scenario  development  or  modification.  The 
support  that  does  exist  consists  primarily  of  databases  describing  typical  U.S. 
and  opposing  force  (OPFOR)  units,  along  with  software  for  creating  modified 
unit  descriptions  and  task  organizations.  Since  scenario  development  is  ex¬ 
tremely  labor  intensive  and  requires  a  high  level  of  expertise  in  large  unit 
operations,  the  lack  of  support  in  automated  systems  for  this  process  has  had 
the  unfortunate  side  effect  of  concentrating  most  command  staff  training  on  a 
few  well  known  scenarios. 
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Although  development  of  training  scenarios  is  currently  something  of  an  art 
form,  with  little  existing  documentation  on  what  should  be  included  or  on  how 
the  situation  can  be  modified  to  meet  particular  training  objectives,  enough  is 
known  about  the  process  to  recommend  the  incorporation  of  tools  to  relieve  the 
scenario  developer  of  some  routine  labor.  In  particular,  terrain  analysis  can 
be  relieved  of  much  tedium  by  incorporating  automated  line-of-sight  fans  and 
three  dimensional  perspective  views  of  the  terrain.  Similarly,  sensor  place¬ 
ment,  weapons  positioning,  and  indirect  fire  coverage  for  the  OPFOR  can  be 
assisted  by  the  use  of  range  fans.  Doctrinal  templates  of  OPFOR  scheme  of 
maneuver  and  defensive  tactics  can  be  provided  to  assist  the  scenario  developer 
in  choosing  initial  positions  and  avenues.  Many  similar  tools  have  already 
been  developed  for  use  with  analytical  wargames  (McGrew,  1980).  Scenario  de- 
velopment  would  be  greatly  facilitated  if  the  simulation  is  based  upon  a  "svs- 


temic"  wargame ,  i.e.,  one  which  is  capable  of  operation  for  extended  periods 


without  controller  input.  The  systemic  model  could  be  run  to  test  scenarios 


to  insure  that  appropriate  force  ratios  are  used.  Further,  the  wargaming  capa 


bility  could  be  used  to  develop  information  on  how  well  a  unit  might  be  ex 


ected  to  perform  under  the  given  conditions  (SAIC,  1985) 
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SECTION  3.  DEVELOP  PERFORMANCE  MEASUREMENT 


Performance  assessment  has  tended  to  receive  insufficient  attention  when 
battle  simulation  systems  are  designed  and  when  they  are  employed.  This  is 
especially  unfortunate  because  the  value  of  the  training  received  is  inextric¬ 
ably  linked  with  the  quality  of  feedback  presented.  Research  by  Thomas, 

Barber,  and  Kaplan  (1983)  shows  that  explicit  incorporation  of  measurement  and 
feedback  in  simulation-based  command  and  control  training  exercises  is 
essential  for  effective  training. 

The  requirements  of  performance  measurement  and  feedback  need  to  be  consid¬ 
ered  early  in  the  design  process  or  even  minimal  system  support  may  be  pre¬ 
cluded.  The  system  must  provide  the  hardware  for  collecting,  storing,  process¬ 
ing,  and  disseminating  performance  data.  Also,  the  software  should  be  designed 
in  such  a  way  as  to  facilitate  the  collection  of  data  on  battle  outcomes, 
status,  resource  usage,  and  other  information  associated  with  training  objec¬ 
tives.  With  the  advent  of  operational  tactical  data  systems,  the  opportunity 
for  on-line  collection  and  analysis  of  staff  performance  data  has  increased 
dramatically,  provided  that  the  linkage  of  the  simulation  system  to  the  tacti¬ 
cal  data  system  is  appropriately  designed. 

Performance  measurement  can  be  addressed  at  several  levels  of  the  command 
staff  environment.  One  classification  scheme  is  to  consider  heirarchial  levels 
of:  (a)  command  post  or  system,  (b)  major  subsystem,  (c)  staff  element,  and 

(d)  individual.  Different  indicators  or  "observables"  are  characteristic  of 
measurement  at  each  level.  Statistics  summarizing  the  outcomes  of  the  simu¬ 
lated  battles  are  useful  at  the  system  and  major  subsystem  (e.g.,  maneuver, 
fire  support,  air)  level.  Products  and  procedures  are  associated  with  evalua¬ 
tion  at  the  staff  element  level.  Knowledge  and  decisions  require  individual 
level  measurements. 

This  discussion  will  cover  basic  measurement  considerations  and  review 
successful  measurement  techniques  for  each  type  of  measure.  Automated  measure¬ 
ment  techniques  and  their  associated  data  and  resource  requirements  will  be 
emphasized. 

Battle  Outcome  Statistics 


This  category  includes  most  of  the  standard  operations  research  techniques 
for  judging  the  outcomes  of  analytical  wargames,  such  as  loss-exchange  ratios 
(LERs),  surviving  maneuver  force  ratio  differentials  (SMFRDs),  and  combat  power 
ratios.  It  also  includes  indicators  of  the  efficiency  of  an  operation  such  as 
consumption  of  supplies  in  various  logistics  categories.  Research  performed  by 
Thomas  (1983)  showed  that,  of  the  most  widely  used  measures  of  this  type,  only 
SMFRDs  accounted  for  a  significant  proportion  of  the  variance  in  expert  judg¬ 
ment  of  mission  accomplishment  for  a  set  of  training  exercises. 

In  subsequent  research  (Thomas  &  Cocklin,  1983),  a  panel  of  experts  evalu¬ 
ated  outcome  measures  from  a  large  number  of  covering  force  operations  and 
assigned  mission  accomplishment  scores.  A  weighted  linear  combination  of 
SMFRDs,  measures  of  territory  lost,  time  the  enemy  was  delayed,  and  accuracy  of 
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intelligence  estimates  at  the  conclusion  of  the  mission  accounted  for  a  much 
higher  proportion  of  the  variance  in  mission  accomplishment  judgments  than 
SMFRD's  alone.  The  regression  model  was  applied  to  a  new  set  of  training 
exercises,  and  was  extremely  accurate  in  predicting  the  mission  accomplishment 
judgments  of  the  same  panel  of  experts.  Further  development,  including  exten¬ 
sion  to  other  missions  and  validation  against  a  larger  sample  of  expert  judg¬ 
ments,  is  required  before  the  regression  models  could  be  used  to  replace  human 
evaluation  of  mission  accomplishment. 

Despite  the  experimental  demonstratic  i  of  how  battle  outcome  statistics 
can  be  used  to  provide  reliable  indicators  of  performance,  they  are  not  suffi¬ 
cient  to  evaluate  staff  performance.  There  are  a  number  of  reasons  for  this. 

First,  there  exists  considerable  doubt  as  to  the  accuracy  of  the  underlying 
models  of  combat  which  provide  the  attrition  results.  This  is  particularly 
true  of  models  which  use  firepower  score  methods  to  estimate  the  relative  abil¬ 
ity  of  a  unit  to  inflict  casualties  on  an  opponent,  because  firepower  scores 
ignore  the  differential  effectiveness  of  various  weapons  systems  against  tar¬ 
gets  of  varying  "hardness".  To  some  extent,  there  is  also  distrust  of  more 
sophisticated  attrition  methodologies  if  they  are  being  used  in  conjunction 
with  unclassified  weapons  effects  data. 

Second,  the  raw  numbers  provided  by  battle  outcome  statistics  are  meaning¬ 
less  in  themselves.  They  must  be  compared  to  standards  of  performance  which 
describe  how  well  a  unit  would  be  expected  to  perform  a  similar  mission  with 
similar  assets  against  a  comparable  opponent.  These  data  do  not  currently 
exist;  they  cannot  be  reliably  obtained  from  training  environments  due  to  the 
variability  of  such  exercises  in  unit  composition,  training  goals  and  methods, 
and  scenarios  played;  and  they  cannot  be  generated  using  current  analytical 
wargames  due  to  the  inherent  speed  limitations  and  lack  of  replicability  of 
man-in-the-loop  simulations. 

Third,  such  data  are  not  a  pure  reflection  of  the  performance  of  a  com¬ 
mander  or  staff.  Outcomes  are  also  influenced  by  the  behavior  of  friendly 

controllers,  data  entry  personnel,  and  the  opposing  force  controllers.  Current 
training  simulations  do  not  automate  the  decision  making  of  subordinate,  sup¬ 
port  or  opposing  elements,  but  rely  upon  very  detailed  model  inputs  provided  by 

training  support  personnel  who  are  often  too  busy  to  provide  appropriate  in¬ 

puts  (Thomas  &  Solick,  1982). 

Finally,  measurement  based  on  battle  outcome  statistics  is  not  well-suited 
to  the  primary  purpose  of  command-staff  training,  which  is  to  diagnose  and 
correct  performance  deficiencies.  This  purpose  is  better  served  by  measurement 
approaches  which  directly  examine  the  tasks  a  staff  performs,  the  way  in  which 
they  do  it,  and  the  products  resulting  from  their  effort. 

As  a  design  issue,  the  use  of  battle  outcome  statistics  as  performance 


measures  increases  requirements  for  simulation  fidelit' 


of  detailed  records,  and  development  of  supporting  techniques  to  acquire 


normative  data  against  which  training  results  can  be  compared. 
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Products  and  Procedures 


The  staff  element  is  the  intermediate  level  in  the  process  heirarchy,  and 
indices  of  staff  performance  primarily  involve  either  products  or  procedures. 
Products  include  items  such  as  estimates,  plans,  orders  and  reports.  Measures 
of  the  products  can  be  on  dimensions  of  timeliness,  accuracy,  understandabil- 
ity,  relevancy  and  completeness.  Procedures  involve  whether  actions  and  prod¬ 
ucts  were  developed  in  accordance  with  prescribed  doctrine  or  Field  SOP. 


Observation  has  been  the  only  technique  commonly  employed  for  evaluation 
of  staff  products  and  procedures  during  simulated  battle.  It  is  usually  done 
informally,  relying  upon  the  participants  and  training  support  personnel  to 
take  note  of  key  events,  apparent  misunderstandings,  and  failures  to  coordinate 
plans.  These  impressions  are  commonly  reviewed  in  a  debriefing  session  at  the 
end  of  the  exercise  (ENDEX),  known  as  an  after  action  review  (AAR).  The  AAR  is 
usually  supplemented  by  a  brief  summary  of  the  status  of  forces  at  ENDEX.  In 
simulation  systems  which  provide  replay  capabilities,  locations,  status,  and 
activities  at  key  points  in  the  battle  are  reviewed  as  the  participating  com¬ 
mander  and  chief  controller  remember  them.  Occasionally,  these  informal  tech¬ 
niques  are  augmented  by  the  presence  of  one  or  more  special  observers  who  may 
record  their  observations  and  evaluations  using  an  ARTEP  task  list  or  other 
mnemonic  such  as  a  list  of  key  information  to  be  presented  in  the  operations 
order  (Olmstead,  Baranick  &  Elder,  1978b).  Some  simulation  centers  have  also 
instrumented  their  simulated  Tactical  Operations  Center  (TOC)  facilities  with 
video  monitors  and  centralized  communication  systems  to  improve  the  effective¬ 
ness  of  observers. 


There  are  at  least  two  opportunities  for  designing  battle  simulations  to 
improve  this  rather  haphazard  approach  to  performance  assessment.  First,  the 
computer  system  can  be  given  a  list  of  key  events  to  look  for  which  indicates 
common  failures  in  planning  and  coordination.  This  list  can  be  used  to  focus 
an  automatic  replay  on  instructive  situations  or  be  provided  as  input  to  a  more 
traditional  AAR.  Examples  of  such  events  include  routing  supplies  through  your 
own  minefield  (indicating  failure  to  coordinate  resupply  and  barrier  plans), 
sealing  off  one  of  your  own  units  with  FASCAM^  (indicating  failure  to  coordi¬ 
nate  maneuver  and  fire  support),  and  cancellation  of  air  strikes  due  to  smoke 
or  dust  over  the  target  area  (failure  to  coordinate  air  support  and  indirect 
fires).  These  and  other  more  subtle  failures  in  planning  or  coordination  can 
be  noted  by  simulation  software  without  relying  upon  an  observer  being  in  the 
right  place  or  recalling  events  that  happened  several  hours  earlier. 


The  second  area  of  opportunity  exists  when  the  staff  being  trained  in  a 
battle  simulation  environment  is  equipped  with  a  tactical  data  system  (TDS)  or 
with  equipment  which  simulates  the  functionality  of  such  a  system.  The  TDS  can 
be  equipped  with  a  "watchdog"  program  to  note  and  record  usage  of  the  system 
for  later  comparison  with  normative  or  doctrinal  data  stored  on  the  training 
device.  The  types  of  events  that  might  be  of  interest  include  the  number  and 
time  of  submission  of  reports  required  by  SOP,  the  categories  of  information 


Family  of  scatterable  mines, 
by  surface  vehicles. 


FASCAM  may  be  implaced  by  artillery,  by  air,  or 
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accessed  by  each  staff  section  during  preparation  of  plans  and  annexes  to  the 
operations  order,  the  frequency  with  which  status  reports  are  requested,  and 
the  delay  in  entering  information  received  via  voice  communications. 


There  has  been  some  experimental  use  of  testing  methods  with  staff  products 
and  procedures.  The  testing  involved  variations  of  probe  methodology,  where  a 
message  is  inserted  into  the  information  flow  of  the  exercise  specifically  to 
generate  some  required  staff  action  so  that  the  type,  quality,  and  timeliness 
of  the  staff  response  can  be  noted  (Carter  &  Patton,  1985).  This  differs  from 
purely  observational  methods  in  that  the  situation  is  deliberately  altered, 
sometimes  in  rather  artificial  ways,  to  generate  behavior  that  would  not  ordi¬ 
narily  occur  in  a  "free  play"  exercise.  Examples  of  such  probes  include  jam¬ 
ming  of  communication,  report  of  a  chemical  weapons  attack,  looting  reports. 


etc.  A  number  of  such  probes  were  developed  for  use  in  manual  simulations. 


5 


Battle  simulations  may  support  the  probe  technique  by  storing  a  list  of 
probe  messages  and  prompting  controllers  to  insert  them  into  the  message  flow 
to  the  TOC  at  specific  times  or  in  conjunction  with  the  occurrence  of  specific 
events  in  the  simulation.  If  the  probes  are  designed  to  generate  some  required 
staff  activity  on  a  tactical  data  system  (TDS),  the  technique  can  be  tied  in 
with  the  previously  suggested  watchdog  program  to  provide  automated  assistance 
in  observation  of  responses  and  response  times. 

Staff  products  and  procedures  are  the  major  emphasis  of  command  post  train¬ 
ing  exercises.  As  such,  they  should  also  be  the  primary  focus  for  automated 
measurement  procedures. 

Knowledge  and  Decisions 

The  individual  performance  level  is  divided  into  knowledge  and  decision 
categories.  Since  knowledge  is  not  directly  observable  and  only  the  results  of 
decision  making  can  be  directly  observed,  measurement  of  individual  performance 
must  rely  upon:  (a)  test  techniques  to  determine  whether  the  individuals  have 
the  knowledge  they  should,  and  (b)  judgments  of  the  quality  of  decisions.  The 
assessment  of  decisions  requires  speculation  about  how  alternatives  would  have 
worked  out;  the  effectiveness  of  a  set  of  decisions  is  seldom  determined. 
Typically  observers  will  only  note  when  a  decision  varies  from  accepted  prac¬ 
tice  or  from  what  they  would  have  done  in  the  same  circumstances. 

The  most  successful  application  of  testing  to  determine  knowledge  is  based 
on  information  flow  methodology  (Kaplan,  1980,  1981).  This  is  a  technique  for 
tracing  the  spread  of  specific  items  of  information  throughout  a  staff  group 
during  planning,  preparation  and  delivery  of  an  operations  order.  It  requires 
a  prior  analysis  of  the  types  of  information  that  each  staff  section  or  subor¬ 
dinate  element  should  receive  on  the  plans,  resources,  and  activities  of  other 
staff  elements.  When  the  mission  is  briefed  to  the  staff  group  undergoing 
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Probes  used  with  manual  simulations  were  intended  only  to  exercise  staff  areas, 
e.g.,  administrative  and  logistics  functions,  that  were  not  well  supported  by 
the  simulation  techniques  employed.  When  combined  with  an  observation  plan, 
they  are  also  useful  for  measurement. 


training,  these  test  elements  of  information  are  presented  individually  to 
different  staff  sections.  After  the  staff  plans  and  presents  its  operations 
order,  they  and  subordinates  are  given  a  written  test  on  the  items  they  re¬ 
ceived  initially,  the  items  they  should  have  received  through  coordination  or 
in  the  operations  order,  and  on  other  items  that  were  inserted  but  not  directed 
to  them,  i.e.,  items  that  they  were  not  required  to  know  to  perform  their  indi¬ 
vidual  functions. 

Attempts  have  been  made  in  experimental  situations  to  extend  the  informa¬ 
tion  flow  method  to  the  mission  execution  phase  of  exercises  (Thomas,  Barber,  & 
Kaplan,  1984;  and  Thomas,  Kaplan,  &  Barber,  1984).  Knowledge  was  tested  in 
each  staff  section  by  requests  from  appropriate  controllers.  Specific  queries 
were  directed  to  key  staff  members  in  the  experiments  by  Thomas,  et  al ,  rather 
than  introducing  significant  events  and  tracing  the  spread  of  information 
throughout  the  staff  as  proposed  by  Carter  and  Patton  (1985).  The  limitation 
on  this  technique  is  on  the  number  of  seemingly  random  queries  that  can  be 
instituted  without  interfering  with  the  attempt  to  maintain  a  realistic  battle¬ 
field  situation.  In  the  cited  studies,  5-6  queries  per  staff  element  were 
inserted.  This  number  is  too  low  for  any  statistical  conclusions  to  be  drawn 
about  individual  performance,  but  the  numbers  aggregated  across  staff  elements 
could  give  fairly  reliable  indications  of  a  staff  group  with  problems  in  inter¬ 
nal  coordination. 

Another  approach  to  determining  the  accuracy  and  timeliness  of  distribution 
of  information  was  explored  in  several  exercises.  It  was  assumed  that  the  in¬ 
formation  posted  on  the  various  situation  maps  maintained  by  different  staff 
sections  was  an  indicator  of  the  knowledge  held  by  the  staff.  By  sampling  that 
information  periodically  and  comparing  it  to  the  "ground  truth"  information 
taken  directly  from  the  battle  simulation,  one  could  obtain  data  on  the  time¬ 
liness  and  accuracy  of  information  dissemination.  In  practice,  the  technique 
did  not  work.  The  basic  practical  difficulty  was  in  obtaining  information 
posted  on  the  maps  and  analyzing  it  in  time  to  be  useful  for  feedback  to  the 
command  group.  Another  difficulty  was  in  interpreting  the  results  obtained. 
There  is  currently  no  doctrine  for  how  much  timeliness  or  how  much  accuracy  is 
enough.  In  results  from  a  division  level  exercise,  it  was  clear  that  the  work¬ 
ing  answer  varied  widely  across  staff  sections.  It  would  be  costly  to  assemble 
enough  normative  data  using  manual  techniques  to  permit  interpretation  of  these 
numbers . 

Though  the  testing  methods  used  to  evaluate  knowledge  were  successful  in 
experimental  tests,  they  are  impractical  to  implement  due  to  the  amount  of 
manpower  required.  This  suggests  another  design  criterion  for  battle  simula¬ 
tions:  they  should  provide  automated  assistance  in  initiating  performance 

tests,  in  collecting  performance  data,  and  in  analytically  developing  standards 
for  performance,  particularly  in  the  area  of  evaluating  decisions.  Two  tech¬ 
niques  are  suggested  here  that  require  attention  in  the  design  of  new  battle 
simulations:  a  comparison  of  the  state  of  the  model  with  the  state  of  the 

staff's  knowledge,  as  represented  in  the  data  base  of  a  TPS,  and  the  develop¬ 
ment  of  wargaming  methods  for  assessing  alternative  decisions. 
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The  first  technique  proposed  is  essentially  an  automation  of  the  comparison 
of  the  "ground  truth"  in  the  model  with  information  "posted"  by  the  staff  in 
their  TDS.  If  implemented  appropriately,  however,  it  can  go  well  beyond  a 
simple  comparison  of  location  and  status  information.  For  example,  if  the 
staff  is  required  to  formally  post  estimates,  these  can  be  analyzed  computa¬ 
tionally  and  compared  to  the  "real  world"  in  the  model. 

The  second  technique,  analytical  wargaming,  is  an  extension  of  the  previous 
proposal  for  generation  of  normative  data  on  outcome  measures.  If  wargames  are 
to  be  used  for  comparison  of  alternative  decisions,  they  must  provide  an  ex¬ 
plicit  representation  of  decision  making.  To  minimize  the  influence  of  con¬ 
troller  activities  on  the  training  exercise  results,  the  model  used  to  drive 
the  training  system  should  also  operate  in  a  systemic  fashion.  Thus  we  recom¬ 
mend  that  the  same  basic  model  should  be  used  for  both  purposes  to  simplify  the 
comparison  of  alternative  decisions. 

Very  fast  systemic  models  of  the  type  needed  are  currently  under  develop¬ 
ment  by  Training  and  Doctrine  Command  Analysis  Center  (TRAC)  (viz.  the  VIC  — 
Vector  In  Commander  —  model)  and  under  the  auspices  of  Defense  Advanced  Re¬ 
search  Projects  Agency  (DARPA),  where  Corps  Battle  Analyzer  (CORBAN)  is  being 
implemented  on  an  experimental  parallel  processor.  Current  results  indicate 
that  VIC  will  run  ten  times  faster  than  real  time,  in  contrast  to  slower  than 
real  time  operation  for  man-in-the-loop  versions  of  analytical  models.  The 
first  parallel  version  of  CORBAN  is  sixty  times  faster  than  its  implementation 
on  a  standard  serial  architecture.  These  results  indicate  that  this  technology 
may  soon  become  a  practical  design  alternative  for  training  simulations. 
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SECTION  4.  DEFINE  SIMULATION  REQUIREMENTS 


There  are  two  fundamental  questions  to  be  answered  in  defining  requirements 
for  a  training  simulation:  (a)  what  to  simulate,  and  (b)  how  to  simulate  it. 
Ideally,  the  answer  should  flow  directly  from  an  analysis  of  training  objec¬ 
tives;  specifically,  who  is  to  be  trained,  on  what  tasks,  under  what  conditions 
and  to  what  standards.  "n  practice,  neither  the  state-of-the-art  in  modeling, 
nor  in  task  analysis  have  permitted  such  a  straightforward  translation  of  re¬ 
quirements.  Previous  simulations  for  training  have  been  designed  upon  the 
basis  of  what  modeling  techniques  and  computer  resources  were  available,  with 
an  implicit  assumption  that  controllers  could  use  manual  simulation  or  role- 
playing  techniques  to  fill  in  the  gaps.  This  evolutionary  approach  to  system 
design  has  had  some  unfortunate  side  effects.  These  will  be  discussed  under 
four  categories:  model  fidelity,  software  maintenance,  exercise  preparation, 
and  control  of  training  conditions.  Suggestions  will  be  presented  for  more 
closely  approximating  the  desired  relationship  between  training  objectives  and 
system  design. 

Model  Fidelity 

General  requirements  for  fidelity  in  a  training  simulation  include  the 
ability  to  portray  those  missions  that  might  be  realistically  assigned  to  the 
unit  being  trained;  the  availability  of  all  major  resources  that  might  be 
under  the  control  of  the  unit;  the  ability  to  portray  realistic  restrictions  on 
resources  and  activities  imposed  by  terrain,  weather,  competition  for  scarce 
assets,  equipment  and  personnel  limitations,  and  the  ability  to  portray  realis¬ 
tic  timing  of  activities,  for  example,  movement,  coordination,  set-up  and 
breakdown  times,  planning  and  implementation  time  required  by  subordinate  and 
support  elements.  Comprehensive  listings  of  the  resources  and  activities  that 
should  be  portrayed  in  staff  training  are  available  in  Miller  and  Bonder  (1982) 
and  Michel  and  Solick  (1983). 

Discussion  of  some  common  omissions  in  developing  systems  is  provided  in 
Thomas  and  Solick  (1982).  For  battalion  and  brigade  level  training  systems, 
nuclear  and  chemical  contamination,  sensor  systems,  movement  of  artillery  and 
air  defense  assets,  counter  battery  and  counter  mortar  fire,  electronic  war¬ 
fare,  construction  time  for  obstacles,  maintenance  of  equipment,  and  medical 
problems  were  usually  not  included  in  models.  Since  then  various  features  from 
this  list  have  been  added  to  developing  simulations,  but  none  of  the  current 
training  systems  have  incorporated  all  of  the  previous  omissions. 

Two  additional  fidelity  issues  impinge  directly  upon  the  scope  of  the  model 
and  the  size  and  type  of  equipment  needed  to  accommodate  it.  First  is  the 
level  at  which  to  simulate  subordinate  units.  The  standard  answer  is  two  eche¬ 
lons  below  the  level  of  the  training  audience.  While  this  answer  is  adequate 
for  maneuver  units,  it  can  lead  to  an  underestimate  of  the  number  of  entities 
to  be  modeled,  and  thus  potentially  to  a  selection  of  inadequate  equipment  and 
to  a  bottleneck  in  data  entry.  In  the  development  of  specifications  for  the 
Corps  Battlefield  Simulation  (SAIC,  1985)  it  was  determined  that  certain  highly 
specialized  support  elements  required  modeling  at  the  company  or  platoon  level. 
This  roughly  doubled  the  number  of  entities  to  be  modeled. 


A  second  basic  fidelity  issue  is  whether  the  training  system  is  to  operate 
on  classified  or  unclassified  data.  Cost  to  incorporate  electronic  shielding 
and  to  provide  secure  communications  is  usually  contrasted  with  the  perceived 
benefit  of  obtaining  realistic  combat  results,  particularly  if  the  results  are 
to  be  used  for  performance  evaluation.  Briefly,  there  is  no  evidence  that  this 
factor  makes  a  difference  in  training.  Other  considerations,  such  as  whether 
the  system  is  intended  to  exercise  classified  scenarios,  will  override  training 
benefit  issues. 

The  basic  lesson  learned  in  the  development  of  models  to  support  training  is 
that  maneuver  and  direct  fire  conflicts  do  not  constitute  a  combined  arms 
training  system.  The  modeling  effort  and  system  resources  required  for  indi¬ 
rect  fires,  air  and  air  defense,  intelligence  and  logistics,  is  at  least  as 
great  as  that  required  for  maneuver.  If  medical  or  maintenance  functions  are 
to  be  included  with  sufficient  fidelity  to  provide  training  in  the  performance 
of  those  functions,  each  of  those  areas  will  require  similar  resources  to  the 
foregoing.  Failure  to  take  these  factors  into  account  has  led  repeatedly  to  a 
system  design  that  required  extensive  modification  (e.g.,  MACE/BABAS)  or  to  an 
unworkable  concept  (e.g.,  the  distributed  CATTS  system  and  CAMMS  II  were  both 
incapable  of  accommodating  the  required  volume  of  messages  over  phone  lines  to 
a  central  processor).  For  the  systems  that  have  been  fielded  despite  a  neglect 
of  combat  support  and  combat  service  support  functions,  there  have  been  consis¬ 
tent  complaints  from  the  field  that  they  require  an  unrealistically  large  num¬ 
ber  of  controllers  to  manually  portray  the  neglected  functions  (Thomas  & 

Solick,  1982;  Barber  &  Solick,  1980;  Kaplan  &  Barber,  1979;  and  Barber,  McGrew, 
Stewart,  &  Andrews,  1979)  and  that  they  fail  to  address  many  of  the  training 
objectives  of  the  principal  staff  members  (Kaplan,  1984;  Kaplan,  1985;  and 
Kaplan  &  Fallesen,  1986) 

Software  Maintenance 


The  evolutionary  approach  to  the  design  of  battlefield  simulations  has 
tended  to  neglect  many  issues  that  affect  the  system's  success  as  a  home  sta¬ 
tion  training  device. 

Continual  changes  in  equipment,  system  capabilities,  and  doctrine  for  force 
design  require  that  a  training  system  be  designed  for  continual  revision.  It 
is  fairly  straightforward  to  include  in  the  initial  design  of  the  training 
system  provisions  for  programmatic  changes  that  will  occur  during  its  design 
life,  such  as  new  weapon  systems.  However,  incremental  improvements  in  speed, 
range,  accuracy  or  resource  consumption  of  systems  frequently  require  data 
base  changes  after  a  training  system  is  developed  and  fielded.  The  need  for 
change  also  arises  as  new  data  become  available  on  lethality  or  vulnerability 
and  as  new  means  are  developed  for  employing  existing  equipment.  Forecasting 
such  changes  based  upon  current  experience  with  design  modifications  can  mini¬ 
mize  the  disruption  in  fielding  training  devices  by  providing  excess  computa¬ 
tional  and  storage  capacity  in  the  original  design.  It  is  essential  to  have 
comprehensive  documentation  of  the  system  for  such  cases,  including  documenta¬ 
tion  of  the  relationship  between  software  models  of  processes  and  associated 
control  staff  activities,  such  as  data  entry,  model  output  interpretation  and 
role  playing  tasks.  Model  maintenance  is  considerably  easier  if  a  structured 


design  technique,  e.g.,  JSD/JSP  (Jackson,  1983),  is  used  to  prepare  a  simula¬ 
tion  for  implementation  and  if  modular  programming  (e.g.,  ADA,  Modula-2)  or 
object-oriented  programming  (e.g., ROSS,  Smalltalk,  LOOPS,  Flavors)  is  used  for 
the  implementation  (Pressman,  1982). 

Exercise  Preparation 

Preparation  for  a  training  exercise  includes  development  or  selection  and 
modification  of  a  scenario  tailored  to  the  unit's  training  objectives,  adapta¬ 
tion  of  a  generic  data  base  to  the  specific  forces  included  in  the  scenario, 
and  preparation  to  carry  out  the  plan  developed  by  the  training  audience. 
Preparations  also  include  training  of  support  staff  to  operate  the  equipment, 
to  plan  the  exercise  in  accordance  with  training  objectives,  and  to  play  the 
roles  of  higher,  subordinate,  and  adjacent  headquarters.  Current  simulations 
usually  provide  one  or  more  standard  scenarios,  a  suggested  program  of  instruc¬ 
tion  for  controllers,  and  user  manuals  which  list  control  functions.  The  most 
recently  fielded  training  system,  ARTBASS,  also  includes  computer-assisted 
instruction  on  operation  of  the  computer  system.  In  general,  however,  little 
is  provided  in  the  nature  of  tools  for  scenario  development  for  data  base  modi¬ 
fication  in  accordance  with  doctrinal  templates,  for  planning  an  exercise  to 
meet  training  objectives,  nor  for  instruction  on  playing  the  role  of  other 
headquarters  elements  (Kaplan,  1985). 

A  complete  system  design  should  include  these  considerations,  since  the 
cost  of  such  preparations  approach  the  cost  of  conducting  the  actual  training 
exercise  (Kaplan  &  Barber,  1979).  Furthermore,  it  may  not  be  most  effective  to 
consider  them  as  low-cost  add-ons  to  be  designed  later,  as  the  equipment  se¬ 
lected  for  the  exercise  may  not  provide  the  most  appropriate  input  and  output 
devices  for  these  other  system  functions.  For  example,  mouse  input  devices  are 
useful  for  high-speed  menu  selection,  but  not  as  good  as  graphics  tablets  for 
entry  of  detailed  graphics  such  as  avenues  of  advance,  phase  lines,  or  control 
measures.  Similarly,  optical  scanning  or  bar-code  readers  might  be  best  for 
setting  up  a  unit  data  base,  but  not  for  initiating  resupply  actions  (Monk, 
1984). 

Control  of  Training  Conditions 

Design  of  simulations  to  accommodate  varying  training  conditions  is  usually 
interpreted  strictly  in  tactical  terms,  e.g.,  varying  geography/terrain,  vary¬ 
ing  weather  conditions,  accommodating  different  unit  types,  or  varying  missions 
and  combat  ratios.  While  these  factors  have  been  shown  to  strongly  influence 
simulated  combat  results  (Thomas  &  Cocklin,  1983)  they  are  not  closely  related 
to  measures  of  the  products  (plans,  estimates,  reports)  that  staff  members 
produce  nor  to  measures  of  the  extent  to  which  they  follow  accepted  procedures 
(Thomas,  Barber  &  Kaplan,  1983).  In  order  to  directly  manipulate  the  diffi¬ 
culty  of  staff  tasks,  a  training  simulation  must  be  able  to  provide  control  of 
message  loads,  amount  and  accuracy  of  information  and  intelligence,  integrity 
of  communications  links,  and  other  factors  directly  influencing  information 
processing  and  decision  making.  Control  of  such  factors  is  important  if  train¬ 
ing  is  to  be  tailored  to  the  level  of  skill  of  the  staff  since  a  major  goal  of 
the  training  program  is  to  create  command  groups  capable  of  functioning  smooth¬ 
ly  under  adverse  conditions.  A  "How  to  Train"  manual  should  be  produced  which 
documents  the  objectives  that  the  system  can  support,  the  resources  needed  to 
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conduct  training  exercises,  the  ways  in  which  exercises  can  be  tailored  to 
specific  deficiencies,  and  the  ways  in  which  the  system  supports  performance 
measurement  and  feedback.  This  manual  should  address  the  composition  of  the 
intended  training  audience  and  the  ways  in  which  supplementary  training  can  be 
provided  to  those  members  whose  function  is  only  marginally  supported  by  the 
simulation.  The  "How  to  Train"  manual  should  be  included  as  on-line  documents 
tion  for  the  system  to  synchronize  the  maintenance  of  software  and  documentati 
on,  to  permit  more  rapid  dissemination  of  changes,  and  to  provide  on-line 
assistance  in  resource  calculations  and  scheduling  of  training  events. 


SECTION  5.  DEFINE  CONTROLLER  TASKS 


Defining  the  tasks  performed  by  the  training  support  personnel  lies  at  the 
heart  of  the  design  process.  Controller  tasks  must  be  explicitly  defined  in 
order  to:  (a)  delimit  the  performance  requirements  of  the  simulation,  (b) 
design  the  interface  between  the  controller  and  the  simulation  and  the  inter¬ 
face  between  the  controller  and  the  players,  (c)  estimate  the  system  manning 
requirements,  and  (d)  develop  the  supporting  documentation  and  training  manu¬ 
als. 


In  current  simulations  there  are  three  fairly  distinct  classes  of  training 
support  personnel:  the  controllers,  the  interactors,  and  the  role  players. 
Controllers  include  dedicated  simulation  center  personnel  to  handle  functions 
that  require  detailed  knowledge  of  the  training  system  and,  in  some  cases, 
members  from  the  training  audience's  parent  headquarters  to  oversee  the  unit 
training.  Interactors  perform  primarily  clerical  tasks,  entering  data  at  the 
direction  of  controllers  and  role  players  and  calling  up  displays  and  status 
reports.  They  provide  the  training  to  role  players  on  the  system  capabilities 
and  limitations.  Role  players  portray  elements  with  whom  the  commanders  and 
staff  must  interact.  They  often  serve  in  their  actual  duty  position;  translat¬ 
ing  orders  into  activities  that  can  be  entered  into  the  computer;  directing 
their  own  (simulated)  subordinates;  and  providing  orders,  information  and  in¬ 
telligence  to  the  training  audience. 

The  discussion  here  will  focus  on  the  control  functions  that  must  be  per¬ 
formed  and  the  system  requirements  for  interfacing  with  these  functions.  The 
emphasis  will  be  upon  identifying  and  making  design  provisions  for  control 
Casks  that  minimize  the  required  number  and  qualifications  of  support  person¬ 
nel.  Three  general  areas  of  controller  functions  will  be  discussed  in  this 
section:  controlling  the  exercise,  controlling  the  model,  and  role  playing. 

Controlling  The  Exercise 

The  exercise  control  function  encompasses  the  planning,  preparation,  moni¬ 
toring,  and  evaluation  of  the  exercise.  Control  personnel  function  as  trainers 
insofar  as  they  control  the  conduct  of  the  exercise,  the  conditions  to  which 
the  training  audience  is  subjected,  and  the  planning  of  performance  evaluation 
and  feedback.  To  respond  to  observed  strengths  or  deficiencies  of  the  trainees, 
controllers  need  to  adjust  the  conditions  and  course  of  the  battle  to  present 
appropriate  training  opportunities.  Also,  controllers  may  be  responsible  for 
performance  measurement,  using  probes,  measures,  observing  staff  performance, 
marking  key  events  for  emphasis  during  after  action  reviews  and  coordinating 
with  other  controllers. 

Setting  Training  Conditions.  It  has  been  observed  that  insufficient  provi¬ 
sions  are  made  for  adjusting  the  conditions  under  which  the  command  staffs 
train  (Fallesen,  Lussier,  &  Garlinger,  1986).  In  a  given  exercise,  controllers 
may  wish  to  change  the  difficulty  of  the  exercise  by  increasing  the  tempo  of 
battle,  removing  key  personnel  suddenly,  or  jamming  communication  equipment. 
Command  groups  need  both  to  train  on  conducting  a  battle  which  follows  prepared 
plans  and  to  react  to  sudden  events  just  as  in  actual  battle. 
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Sometimes  the  command  group  should  receive  clear  and  accurate  intelligence 
from  above  and  below,  and  at  other  times  their  information  should  be  vague  and 
distorted.  In  short,  simulation  systems  need  to  be  flexible  enough  to  allow 
play  under  a  wide  variety  of  conditions  (Olmstead,  Baranick,  &  Elder,  1978a). 
The  choice  of  conditions  should  depend  on  the  entrance  level  of  training  of  the 
unit,  the  specific  training  objectives  of  the  exercise,  and  on  the  course  of 
play  during  the  exercise.  System  developers  need  to  be  aware  of  this  control¬ 
ler  function  and  provide  methods  to  put  the  control  of  training  conditions  in 
the  hands  of  the  controllers.  Systems  should  be  adaptive  to  the  performance  of 
the  command  group,  recognize  areas  in  which  more  training  is  necessary  and 
guide  the  play  of  the  exercise  in  a  manner  which  presents  the  appropriate 
training  opportunities  (Shaket,  Ben-Basset,  Madni,  &  Leal,  1979).  Training 
systems  must  be  designed  with  sufficient  flexibility  to  allow  adaptive  control 
of  conditions. 


Evaluating  Performance.  Another  overall  responsibility  of  controlling  the 
exercise  is  evaluation.  Evaluation  tasks  have  been  performed  by  interactors  as 
well  as  controllers  and  infrequently  by  independent  observers  without  other 
control  tasks.  The  measurement  of  the  performance  of  the  command  staff  was 
covered  in  section  3  of  this  report,  but  it  cannot  be  overemphasized;  the  pres¬ 
entation  of  useful  feedback  is  critical  to  the  success  of  the  exercise.  Not 
only  must  the  feedback  be  accurate,  it  must  be  accepted  by  the  training  audi¬ 
ence.  This  means  that  controllers  participating  in  evaluations  must  be  able  to 
support  their  conclusions,  ideally  through  an  objective  system  of  performance 
measurement. 

An  after  action  replay  function  and  combat  outcome  summaries  are  both  tech¬ 
niques  designed  to  assist  in  providing  feedback.  Neither,  however,  explicitly 
identifies  the  specific  strengths  or  deficiencies  of  the  training  unit.  With¬ 
out  further  development  of  a  performance  measurement  technology,  these  methods 
do  not  allow  an  effective  assessment  of  the  command  group.  However,  the  incor¬ 
poration  of  performance  measurement  techniques  will  also  impose  additional 
tasks  upon  the  training  support  staff,  additional  requirements  for  training 
controllers  to  perform  these  tasks,  and  additional  control  staff  coordination 
requirements.  These  considerations  must  be  included  in  the  workload  and  sys¬ 
tem  manning  analysis. 

ARI  is  conducting  research  in  the  area  of  command  staff  performance  meas¬ 
urement  and  will  be  able  to  provide  more  specific  assistance  in  the  future.  At 
present  a  few  basic  suggestions  can  be  made.  First,  the  controller  workload 
should  not  be  so  heavy  as  to  prohibit  the  possibility  of  measurement  and  eval¬ 
uation  functions.  Second,  there  should  be  some  provision  made  for  the  presence 
of  evaluators,  to  include  the  ability  to  view  events  occurring  in  the  model, 
traffic  on  the  communication  nets,  and  activity  of  the  training  audience. 

Third,  no  matter  how  rudimentary,  a  start  must  be  made  on  providing  automated 
assistance  to  the  measurement  problem.  The  system  could  be  designed  to  iden¬ 
tify  and  flag  occurrences  from  a  previously  identified  set  of  typical  errors  at 
a  given  echelon,  to  request  staff  products,  to  prompt  controllers  to  insert 
specific  behavioral  probes,  and  to  request  that  the  controllers  input  evalu¬ 
ative  information  at  critical  points  during  the  exercise. 
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Controlling  The  Model 

Model  control  involves  the  translation  of  the  directives  of  the  training 
audience  into  commands  for  entry  to  the  computer  and  control  of  the  operation 
of  the  battle  models.  The  tasks  can  be  as  tedious  as  arranging  for  the  move¬ 
ment  of  a  battalion,  platoon  by  platoon,  and  as  creative  as  simulating  the 
effects  of  resources  which  are  not  modeled,  e.g.,  achieving  the  same  effects  as 
a  demolition  team  through  "phantom"  airstrikes. 

The  controllers  must  be  able  to  implement  the  orders  of  the  commander  and 
staff  without  major  discrepancies  or  omissions  and  must  do  it  without  lengthy 
delays.  Imagine,  for  example,  that  troop  movements  were  achieved  in  an  accu¬ 
rate  and  timely  manner,  but  either  engineer  operations  or  fire  missions  were 
greatly  delayed  or  altered.  Such  a  situation  frustrates  the  training  audience 
as  it  destroys  the  synchronization  of  their  forces  and  the  effectiveness  of 
their  plans. 

A  primary  purpose  of  any  training  exercise  is  to  identify  deficiencies  in 
the  behavior  of  the  training  audience.  The  commander  and  staff,  however,  typi¬ 
cally  find  it  very  easy  to  see  the  deficiencies  of  the  system  and  very  diffi¬ 
cult  to  see  their  own  faults.  Some  tolerance  for  interactor  error  exists  and 
can  be  considered  as  "fog  of  battle."  It  would  be  better,  however,  if  this 
"fog  of  battle"  effect  was  intentional  and  was  guided  by  the  training  objec¬ 
tives.  As  indicated  above  it  is  sometimes  desirable  to  conduct  training  under 
the  conditions  of  a  perfectly  efficient  set  of  simulated  subordinate  units 
while,  in  other  cases,  a  less  efficient  group  of  "subordinates"  would  be  better 
to  train  the  monitoring  and  directing  aspects  of  command  and  control.  In  any 
event,  when  the  controllers  cannot  effectively  control  the  model,  it  is  almost 
certain  that  the  training  audience  will  see  the  model/interactor  difficulties 
as  an  excuse  for  all  the  problems  of  the  exercise. 

In  part,  the  solution  involves  greater  attention  to  the  design  of  the 
interactor-computer  interface,  and  a  strong  commitment  to  human  factors  engi¬ 
neering  (Monk,  1984).  Prior  to  design  of  the  user  interface,  system  developers 
should  develop  a  list  of  criteria  and  consult  with  human  factors  experts  on  how 
the  desired  performance  may  be  attained.  Basic  issues  to  consider  include 
sharing  of  displays,  color  versus  monochrome  display  requirements,  multiple 
displays  versus  multiple  windows,  and  use  of  iconic  and  nomographic  techniques. 
Attention  should  be  given  to: 

•  Providing  a  clear,  easy-to-learn  (self  teaching)  network  of  commands  and 
menus  without  employing  abstract  coding. 

•  Using  formats  and  terminology  which  are  natural  to  the  military  user. 

•  "Bombproofing"  data  entry  techniques  so  that  the  user  is  not  allowed  to 
make  irreversible  errors. 


•  Allowing  recovery  from  error  which  entails  a  minimum  of  reentry. 

•  Extensive  employment  of  feedback,  acknowledgements  of  instructions, 
prompts,  and  warnings. 
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•  Use  of  an  appropriate  set  of  input  devices  and  creatively  designed  soft¬ 
ware  to  minimize  workload. 

•  Highlighting  of  critical  or  urgent  messages  and  actions. 

A  second  avenue  of  approach  to  the  goal  of  reducing  the  model  control  tasks 
is  through  more  extensive  automation.  Using  operations  research  or  artificial 
intelligence  (AI)  techniques,  the  computer  can  assume  increasing  amounts  of  the 
control  workload  (Shannon,  1975,  Barr  &  Feigenbaum,  1982).  An  example  is  the 
current  ability  of  some  systems  to  employ  route  optimization  routines.  Control 
of  the  opposing  force  is  particularly  open  to  AI  application  because  there  is 
no  requirement  for  communication  or  coordination  with  members  of  the  training 
audience . 

The  design  goal  must  be  to  reduce  the  model  control  tasks  to  an  absolute 
minimum,  in  terms  of  both  physical  and  mental  workload.  Requirements  for  com¬ 
puter  expertise  must  be  reduced  to  the  point  that  controllers  drawn  from  the 
unit  being  trained  can  operate  the  equipment  and  can  translate  computer  output 
into  tactical  messages  after  a  brief  training  session.  Workload  reduction  will 
reduce  the  number  of  controllers  required  by  eliminating  those  who  perform  only 
data  entry  functions,  will  reduce  controller  training  time,  and  will  free  con¬ 
trollers  to  perform  other  valuable  tasks.  Realistic  goals  must  be  set  for  the 
size  of  the  control  staff  and  for  the  amount  of  exercise  preparation  required. 
The  additional  analysis  and  software  development  costs  this  approach  entails 
will  be  recovered  through  increased  efficiency  in  conducting  training. 

Because  units  must  train  as  they  would  fight,  future  battle  simulation 
exercises  will  need  to  incorporate  tactical  data  systems  such  as  the  Maneuver 
Control  System,  which  is  being  fielded  at  echelons  from  battalion  through 
corps.  The  introduction  of  tactical  data  systems  to  command  and  staff  exer¬ 
cises  has  the  potential  to  significantly  reduce  the  controller's  model  control 
workload.  Information  will  need  to  flow  between  the  players  and  the  computer 
in  both  directions,  with  the  results  of  the  computer  modeling  process  automat¬ 
ically  updating  the  tactical  data  system's  database,  and  command  group  direc¬ 
tives  helping  to  control  the  operation  of  the  model.  The  role  playing  workload 
may  also  be  reduced  by  automatic  generation  and  digital  transmission  of  peri¬ 
odic  reports  required  by  field  SOP  (SAIC,  1985). 

In  summary,  the  tasks  for  controlling  the  model  need  to  be  reduced  to  a 
minimum.  If  the  tasks  are  too  numerous,  time-consuming  or  complex,  the  train¬ 
ing  effectiveness  of  the  system  will  be  greatly  impaired.  Representation  of 
battle  events  should  not  be  at  a  level  of  detail  below  that  necessary  to  pres¬ 
ent  a  realistic  picture  to  the  training  audience.  If  the  representation  is 
much  beyond  the  minimum  necessary,  controller  workload  may  suffer.  Finally, 
simulations  need  to  be  designed  so  that  the  link  with  tactical  data  systems  at 
least  eases  the  controller  workload  rather  than  intensifies  it.  Efforts  to  en¬ 
hance  the  human  factors  design  or  to  automate  the  controller  functions  are  not 
frills  but  are  vital  given  the  conditions  under  which  the  systems  are  used  in 
the  field. 


I 


asagsgsgg i 


N’CsSXV 


on 


"  JTi.v^’vvs  yr  v*~~ 


Role  Playing 


Controllers  must  play  the  roles  of  key  personnel  in  units  which  are  higher, 
subordinate,  and  adjacent  to  the  training  unit.  The  role  playing  function 
serves  to  insulate  the  training  audience  from  the  computer  and  to  allow  train¬ 
ing  in  required  communications.  Typically,  subordinate  unit  commanders  are 
played  by  people  brought  to  the  exercise  by  the  training  unit  and  are  often  the 
actual  subordinate  unit  commanders.  Occasionally,  the  higher  headquarters 
assign  personnel  to  the  exercise  to  act  in  various  positions  from  that  head¬ 
quarters.  More  often,  however,  controllers  from  the  simulation  center  portray 
the  roles  of  higher  headquarters.  While  adjacent  units  need  to  be  included  in 
the  scenario,  it  is  rare  to  have  role  players  in  these  positions;  information 
normally  obtained  from  adjacent  unit  commanders  can  be  obtained  from  the  higher 
headquarters  role  players  without  a  significant  loss  of  realism. 

Subordinate  Role  Playing.  Subordinate  unit  role  players  must  keep  them¬ 
selves  apprised  of  the  tactical  situation,  manage  simulated  subordinate  and 
support  units  in  accordance  with  the  plans  and  orders  of  the  command  group,  and 
generate  an  information  stream  of  realistic  tactical  messages  based  on  the 
output  of  the  simulation.  The  latter  function  is  not  well  supported  by  the 
current  generation  of  battle  simulations.  In  particular,  the  output  of  auto¬ 
mated  simulations  appears  to  compromise  realism  of  the  message  traffic  by  pro¬ 
viding  too  much  military  intelligence  and  not  enough  detail.  For  example,  a 
sensor  report  might  include  a  complete  opposing  force  unit  designation  and  an 
exact  center-of-mass  location,  whereas  a  realistic  report  might  provide  an 
estimate  of  a  number  of  vehicles  sighted,  type  of  vehicles,  approximate  loca¬ 
tion  and  direction.  Role  players  are  required  to  perform  this  translation. 
Preliminary  results  from  work  in  progress  at  Fort  Leavenworth  ARI  Field  Unit 
indicates  that  this  translation  task  is  difficult  to  perform.  The  difficulty 
is  compounded  in  the  context  of  an  exercise  with  the  demands  of  the  role 
player's  other  duties  and  the  obligation  to  report  to  one's  actual  superiors. 

One  possible  solution  would  be  to  automate  the  production  of  message  traf¬ 
fic.  This  would  insure  that  input  to  the  training  audience  would  accurately 
portray  events  in  the  battle,  would  not  reveal  unrealistic  amounts  of  intelli¬ 
gence,  and  would  conform  to  standard  military  format.  The  controller  workload 
would  also  be  reduced.  It  has  been  argued  that  the  role  players  are  receiving 
training  benefit  by  composing  the  messages  themselves,  however,  in  the  current 
circumstances  this  argument  seems  to  have  dubious  merit.  Taking  ground-truth 
information  and  dirtying  it  up  for  transmission  to  the  next  higher  echelon  is 
not  a  task  in  which  Army  officers  need  training. 

An  alternative  solution  to  the  translation  problem  is  to  redesign  the  in¬ 
terface  between  the  subordinate  unit  commander  and  the  computer.  Displays  of 
ground-truth  information  could  be  avoided.  Instead,  at  the  subordinate  unit 
commander  stations  (or  other  functional  area  stations  such  as  log/admin,  air, 

Oi  fire  support),  the  role  player  would  have  available  only  that  information 
which  he  could  normally  be  expected  to  have,  either  through  the  activity  of  his 
staff  and  subordinates  or  through  first-hand  observation.  Current  simulations 
have  terrain  appreciation  capability  that  portray  a  3-dimensional  view  from  any 
location  (McGrew,  1980),  given  that  digitized  terrain  data  is  available.  The 


i 


l 


computer  would  need  to  supply  a  further  description  of  what  battle  events  the 
role  player  should  see  from  the  ground,  and  to  provide  displays  of  the  informa¬ 
tion  which  should  be  available  to  the  role  player. 

Although  replacing  ground-truth  computer  representations  of  the  battle  with 
a  realistic  interface  between  the  role  player  and  the  computer  would  require  a 
major  software  development  effort,  there  would  be  many  significant  advantages: 

•  The  training  audience  could  be  supplied  with  a  realistic  message  stream 
which  is  generated  in  a  natural  manner. 

•  Errors  by  role  players  in  portraying  the  situation  would  be  reduced. 

•  Role  players  would  receive  practice  in  skills  which  more  closely  match 
their  usual  military  assignments. 

•  The  interface  could  be  used  to  establish  an  effective  simulation  of  the 
unit  commander  "going  forward"  to  see  the  battle  and  personally  direct  subordi¬ 
nates  . 


Many  of  the  controllers  are  in  a  position  to  receive  considerable  training 
benefit  from  the  exercise.  This  potential  can  be  realized  by  purposely  design¬ 
ing  the  functions  performed  by  subordinate  elements  of  the  command  group  to 
resemble,  as  much  as  possible,  the  tasks  on  which  they  are  to  be  trained. 

Higher  Echelon  Role  Players.  The  role  players  portraying  higher  echelon 
personnel  must  work  on  the  scenario  development,  present  the  initial  briefings 
and  operations  order,  coordinate  with  the  various  functional  areas,  and  make 
decisions  regarding  the  distribution  of  higher  echelon  assets  and  the  dissemi¬ 
nation  of  intelligence.  Higher  headquarters  role  players  are  usually  one  or 
more  staff  officers  from  the  training  unit's  actual  higher  headquarters  or  are 
controllers  who  work  at  the  simulation  center.  In  the  latter  case,  they  may  be 
called  upon  to  portray  positions  with  which  they  have  little  experience. 

There  are  several  areas  in  which  the  system  might  support  the  higher  role 
players,  for  example,  by  aiding  in  scenario  development,  preparing  specialized 
functional  information  packages,  and  by  generating  messages  to  be  inserted 
during  play.  One  observation  concerning  higher  echelon  play  is  that  the  role 
players  provide  much  of  the  support  of  the  higher  echelon,  i.e.,  providing 
intelligence  and  combat  assets,  but  rarely  make  the  demands  of  the  higher  eche¬ 
lon,  i.e.,  additional  taskings  such  as  primary  information  requirements  (PIR), 
other  information  requirements  (OIR),  or  required  standard  reports  (Thomas  & 
Solick,  1982).  The  load  on  the  training  audience  would  be  more  realistic  if 
additional  requirements  were  placed  by  the  higher  echelon.  The  simulation 
could  incorporate  such  taskings  or  prompt  the  role  players  to  demand  scheduled 
reports,  increasing  the  requirements  on  the  command  group. 

In  summary,  role  play  is  an  important  aspect  of  the  exercise  and  can  mean 
the  difference  between  a  command  group  training  in  a  realistic  situation  under 
realistic  constraints  and  a  command  group  which  is  merely  playing  a  battle  game 
against  a  computer.  Neither  the  design  of  current  simulations  nor  the  training 
currently  given  to  the  controllers  provides  sufficient  support  for  this  impor- 
i  tant  function. 


Role  playing  must  be  more  realistic  than  current  practice.  The  message 
traffic  to  the  training  audience  can  be  enhanced  by  automated  production.  An 
improved  computer  interface  needs  to  present  more  realistic  information  to  the 
role  players  in  a  more  usable  form.  The  higher  echelon  needs  to  be  enhanced  so 
all  required  reports  and  additional  taskings  are  included. 


Summary 


In  the  past  the  model  control  task  has  received  the  major  portion  of  atten¬ 
tion  in  the  design  process.  Not  only  is  it  necessary  for  the  operation  of  the 
system  but  it  is  the  function  which  designers  naturally  associate  with  the 
controllers.  Role  playing  is  recognized  during  the  design  phase,  but  it  is 
generally  given  little  or  no  automated  support.  Other  controller  functions 
such  as  controlling  the  exercise  and  performance  evaluation  typically  are  ig¬ 
nored  comDletelv  until  ODerational  testins  of  the  svstem.  For  battlefield 


SECTION  6.  ESTIMATE  CONTROLLER  WORKLOAD 


Simulation  systems  are  developed  in  accordance  with  performance  goals, 
which  specify  the  required  operating  capabilities  of  the  system,  and  staffing 
goals,  which  indicate  the  desired  limit  of  operator  and  controller  person¬ 
nel.  Working  within  the  performance  and  staffing  constraints,  the  system  de¬ 
signer  should  compile  a  complete  description  of  all  the  tasks  performed  by 
controllers,  should  apportion  these  tasks  between  individual  controller  posi¬ 
tions,  and  specify  the  skill  requirements  for  each  position.  The  establishment 
of  manning  requirements  is  normally  an  iterative  process,  involving  redesign 
until  both  the  skill  requirements  and  workload  levels  for  each  proposed  con¬ 
troller  position  are  reasonable. 

The  resulting  estimates  of  the  number  of  controllers  needed  and  the  des¬ 
cription  of  tasks  they  must  perform  drive  the  hardware  design  requirements  for 
controller  workstations.  In  existing  simulations,  several  controllers  are 
assigned  to  each  workstation  with  data  entry  devices  and  situation  map  displays 
usually  shared  by  two  or  three  controllers.  The  tradeoff  between  cost  of 
equipment  and  efficiency  of  control  must  be  considered.  Cost  has  been  the  de¬ 
termining  factor  for  existing  simulations,  but  equipment  costs  have  been  re¬ 
duced  to  the  point  that  efficiency  may  now  predominate.  If  model  control  tasks 
are  distributed  to  the  role  players,  it  is  preferable  to  equip  each  individual 
with  his  own  input  and  display  devices.  It  may  be  necessary  to  develop  pre¬ 
liminary  cost  and  staffing  estimates  for  both  approaches. 

Complete,  well-documented  analysis  of  control  personnel  requirements  is  the 
exception  rather  than  the  rule  in  battle  simulation  development,  primarily 
because  of  the  evolutionary  approach  of  incremental  refinement  to  existing 
simulations.  A  notable  exception  is  the  requirements  analysis  performed  ior 
the  Corps  Battlefield  Simulation  (SAIC,  1985).  The  recommendations  made  in 
this  report  are  primarily  based  upon  that  effort  and  upon  lessons  learned  in 
the  MACE  Concept  Evaluation  Program  (Thomas  &  Solick,  1982). 

The  process  of  determining  manning  requirements  is  presented  in  Figure  3. 
The  initial  step  involves  an  analysis  of  the  component  processes  of  the  model, 
in  which  a  large  and  detailed  list  is  compiled  of  all  the  functions,  tasks  and 
subtasks  which  must  be  accomplished  in  the  training  systems.  Examples  of  such 
an  analysis  may  be  found  in  Miller  &  Bonder  (1982).  The  task  list  should  in¬ 
clude  all  functions  of  the  simulation,  including  the  model  control,  role  play, 
and  evaluation  functions. 

Each  of  the  subtasks  must  be  allocated  to  either  the  controllers  or  the 
computer.  In  general,  this  allocation  should  be  guided  by  a  comparison  of  the 
human  and  computer  strengths  and  weaknesses.  The  ability  to  make  complex  in¬ 
ferences,  to  find  similarities,  and  to  hear,  speak,  and  understand  are  human 
strengths.  The  ability  to  store  and  retrieve  large  amounts  of  information  and 
make  fast  and  accurate  calculations  are  computer  strengths  (Woodson,  1981). 
Further  analysis  of  the  tasks  may  be  helpful  in  making  the  allocation  decision 
(TRADOC  PAM  351-4(T)).  Some  aspects  of  the  tasks  which  may  be  considered  are: 
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Figure  3.  Steps  in  determining  staffing  requirements. 
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•  The  action  performed  -  abstract  the  essential  actions  required. 


•  Skills  required  -  differentiate  between  clerical  activities  and  those 
which  require  judgment  or  perspective. 

•  Knowledge  required  -  indicate  the  military  knowledge  required  and  what 
functional  specialties  might  be  necessary. 

•  Input  required  -  identify  the  characteristics  of  the  input  requirements 
and  source  rather  than  the  specifics  of  current  practice.  Indicate  whether  the 
information  is  gained  "shortly"  before  or  "long  before"  the  task  period  begins. 

•  Output  required  -  as  with  input,  aim  for  the  characteristics  of  require¬ 
ments  and  destinations. 

•  Preceding  activities  -  distinguish  between  the  required  events  which 
precede  task  performance  and  those  which  make  logical  sense  or  are  preferred 
but  are  not  strictly  necessary. 

•  Timing  -  tell  when  the  task  should  occur,  what  kind  of  delays  may  occur, 
what  time  constraints  exist,  what  are  realistic  delays.  How  frequently  do 
tasks  occur?  Note  tasks  which  must  be  performed  concurrently. 

•  Potential  difficulties  -  machine  breakdown  or  backlog,  controller  over¬ 
loading,  etc. 

•  Potential  errors  -  identify  potential  sources  and  the  impact  of  errors. 

Additionally,  the  functions  concerned  with  evaluation  of  the  training 
unit's  performance  need  special  consideration.  The  evaluation  tasks  are  diffi¬ 
cult  and  seem  to  require  much  human  assistance.  They  also  are  tasks  which  can 
be  ignored  without  "crashing"  the  exercise  but  are  vital  to  attainment  of  the 
goals  of  the  exercise.  Evaluation  tasks  need  to  be  given  sufficient  machine 
support  to  insure  that  they  receive  the  controllers'  attention. 

Once  the  tasks  to  be  accomplished  by  controllers  have  been  analyzed,  a 
preliminary  list  of  controller  positions  can  be  established  based  on  the  staff¬ 
ing  goal,  on  the  functional  specialities  for  role  playing,  and  on  the  essential 
model  control  functions.  The  preliminary  manning  scheme  is  completed  by  taking 
each  of  the  tasks  required  and  assigning  it  to  one  of  the  proposed  controller 
positions . 

The  manning  scheme  must  be  analyzed  to  discover  weak  points,  where  either 
the  skill  or  workload  requirements  are  too  high.  Two  general  principles  should 
guide  the  analysis.  First  it  is  better  to  conduct  experiments,  observe  current 
exercises,  or  use  survey  or  historical  data  than  it  is  to  simply  guess.  Sec¬ 
ond,  the  estimates  should  be  conservative,  leaving  sufficient  margin  for  error. 
There  will  be  tasks  that  have  been  overlooked,  activity  at  peak  levels  which 
was  not  anticipated,  and  personnel  who  work  with  less  than  100  percent  effi¬ 
ciency  or  who  lack  skills. 


One  aspect  of  workload  estimation  which  needs  to  be  addressed  is  the  fre¬ 
quency  of  occurrence  of  the  tasks.  This  includes  both  the  number  of  times  the 
task  is  performed  over  the  exercise  (or  per  day)  and  the  peak  frequency,  the 
highest  rate  at  which  the  task  must  be  performed.  Estimation  can  be  assisted 
by  analyzing  doctrinal  material  or  unit  SOPs  or  by  observing  current  simula¬ 
tions.  For  example,  message  traffic  for  a  controller  position  can  be  estimated 
by  determining  the  number  of  SOP  reports  per  transmission  mode  (voice,  digital, 
or  written)  for  that  position  and  inflating  the  totals  to  account  for  non-SOP 
queries  and  replies.  Observing  actual  exercises  at  that  echelon  may  help  to 
determine  the  percentage  of  SOP  to  non-SOP  reports,  the  average  duration  of 
each  transmission  per  mode,  and  the  peak  message  rate.  A  similar  analysis  can 
be  done  for  other  events  such  as  movement  orders  or  fire  missions.  When  using 
data  from  current  simulations,  it  should  be  kept  in  mind  that  the  data  may  not 
be  ideal.  They  represent  the  best  that  can  be  done  in  the  current  generation 
of  simulations  and  do  not  necessarily  reflect  appropriate  levels  for  an  accu¬ 
rate  representation  of  battle. 

Estimates  must  be  made  of  the  time  required  for  controllers  to  perform  each 
task.  For  data  base  entry  and  retrieval  tasks,  one  of  the  best  ways  to  accom¬ 
plish  this  is  to  build  a  mock-up  or  prototype  of  the  man-machine  interface 
early  in  the  design  process.  Experiments  with  the  interface  prototype  will 
serve  to  both  improve  the  design  efficiency  and  provide  workload  estimates  for 
the  individual  tasks.  Additionally,  potential  data  entry  bottlenecks  will  be 
revealed.  The  interface  tests  should  always  be  conducted  with  inexperienced 
personnel  to  insure  that  excessive  controller  requirements  are  not  built  into 
the  system. 

Besides  contributing  to  the  estimates  of  controller  workload,  measures  of 
the  time  required  to  perform  the  various  tasks  are  necessary  to  assess  adequacy 
of  '■he  simulation.  Most  of  the  tasks  can  be  considered  time-critical,  in  that 
excessive  delays  will  seriously  impair  the  realism  of  the  exercise.  For  exam¬ 
ple,  when  indirect  fires  are  called  for,  either  too  responsive  or  too  sluggish 
artillery  action  will  provide  unrealistic  training  conditions.  In  general, 
realistic  delays  can  be  built  into  the  simulation  where  needed.  Data  entry 
bottlenecks,  or  other  input  delays  in  time-critical  tasks,  however,  indicate 
that  the  interface  must  be  redesigned  for  more  efficient  operation,  more  per¬ 
sonnel  must  be  assigned  to  the  task,  or  that  greater  portions  of  the  task  must 
be  automated. 

The  manning  scheme  can  be  assessed  by  considering  each  position,  determin¬ 
ing  whether  the  average  and  peak  workload  levels  and  the  skill  and  knowledge 
requirements  are  reasonable.  The  number  of  control  personnel  can  be  reduced  by 
shifting  tasks,  combining  role  play  with  model  control  and  evaluation  tasks, 
"doubling  up"  of  roles  or  making  other  adjustments  to  the  extent  that  military 
knowledge  requirements  permit.  The  staffing  goal  needs  to  be  met,  of  course, 
and  also  some  survey  of  the  units  which  will  ultimately  use  the  system  must  be 
made.  It  should  be  determined  whether  the  units  are  willing  to  provide  suffi¬ 
cient  numbers  of  persons  with  appropriate  skill  levels  to  support  the  exercise. 
If  it  is  determined  that  planned  controller  levels  cannot  be  reached,  the  proc¬ 
ess  must  be  iterated,  starting  back  at  the  stage  where  tasks  are  allocated  to 
the  computer  or  the  controllers  and  testing  alternative  allocations  and  task 
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SECTION  7.  CONCLUSIONS  AND  SUMMARY  RECOMMENDATIONS 


This  section  recapitulates  the  main  points  of  the  previous  sections.  Most 
of  the  recommendations  presented  are  consistent  with  the  current  evolutionary 
development  approach,  though  increased  emphasis  is  placed  on  analysis  of  the 
tasks  of  both  the  training  audience  and  the  training  support  staff. 

Training  Objectives 

Collection  and  analysis  of  tasks,  conditions  and  standards  is  a  necessary 
precursor  to  development  of  a  training  system.  The  ARTEP  manuals  are  a  good 
starting  place  but  further  development  of  training  objectives  is  necessary. 
Some  personnel  typically  included  in  the  training  audience  are  not  provided 
adequate  training  opportunities.  It  is  recommended  that  a  way  to  provide  ade¬ 
quate  training  be  found  or  these  people  be  removed  from  the  training  audience. 

Tools  should  be  provided  for  development  and  modification  of  scenarios. 
Included  with  the  tools  should  be  on-line  help  facilities  and  documentation  of 
how  the  tools  should  be  used.  If  systemic  wargaming  is  incorporated  as  pro¬ 
posed,  the  model  should  be  adapted  for  testing  and  modification  of  scenarios. 

Performance  Measurement 


Numerous  recommendations  were  made  in  section  3  for  various  levels  of  sup¬ 
port  for  diagnosis  of  deficiencies  and  provision  of  feedback.  These  include: 

•  Develop  templates  for  instituting  the  information  flow  methodology,  to 
be  filled  in  with  specific  items  of  information  from  the  scenario.  Document 
the  methodology  and  the  sources  of  information  in  the  data  base  that  are  to  be 
used . 

•  Develop  a  list  of  probes,  along  with  a  means  for  automatically  notifying 
appropriate  controllers  to  insert  them,  either  at  pre-set  times  or  in  response 
to  simulation  events. 

•  Provide  automatic  detection  of  events  based  on  common  errors  that  indi¬ 
cate  failures  in  staff  planning  or  coordination. 

•  Develop  normative  data  from  model  runs  for  interpretation  of  mission 
accomplishment  data. 

•  Develop  a  watchdog  program  for  the  staff's  tactical  data  system  to  track 
preparation  and  delivery  of  reports. 

•  Develop  procedures  to  compare  the  ground  truth  data  in  the  training 
system  data  base  with  the  staff's  picture  of  the  battle  as  reflected  in  the 
tactical  data  system. 

•  Develop  analytical  wargaming  procedures  to  evaluate  alternative  deci¬ 
sions. 
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Simulation  Requirements 

Beyond  basic  considerations  of  resources  to  portray,  resource  limitations, 
and  level  of  detail  (i.e.,  model  fidelity  )  these  additional  areas  were 
discussed : 

a.  Software  maintenance  was  discussed  as  a  basic  factor  in  a  system's 
success  as  a  home  station  training  device.  Comprehensive  documentation  of  the 
system  and  use  of  modular  programming  techniques  were  recommended. 

b.  The  training  system  should  be  designed  to  minimize  exercise  preparation 
effort.  Scenario  development,  controller  training,  and  data  base  modification 
are  among  the  most  labor-intensive  aspects  of  exercise  preparation. 

c.  Emphasis  should  be  placed  on  training  to  overcome  obstacles  to  effec¬ 
tive  information  processing  and  decision  making.  This  implies  that  control  is 
needed  over  message  loads,  accuracy  of  information,  integrity  of  communications 
and  other  factors  which  directly  affect  the  difficulty  of  staff  work.  A 
"How-to-Train"  manual  should  be  included  as  on-line  documentation. 

Define  Control  Tasks 


Three  general  areas  of  controller  function  were  discussed:  exercise  con¬ 
trol,  model  control,  and  role  play.  Control  functions  required  by  various 
performance  measurement  techniques  were  discussed.  These  impose  system  re¬ 
quirements  for  communication  among  controllers,  observation  of  the  training 
audience  and  interfacing  with  various  automated  measurement  techniques. 
Minimization  of  model  control  functions  was  recommended,  either  through  devel¬ 
opment  of  a  systemic  model,  through  close  attention  to  human  factors  engineer¬ 
ing  of  the  user  interface,  or  through  automation  of  low  level,  repetitive 
controller  decision  making  functions.  Additional  support  for  role  playing  was 
discussed  under  two  alternatives:  automate  production  of  message  traffic,  or 
design  the  simulation  interface  to  supply  the  information  that  subordinate  unit 
commanders  would  ordinarily  have  through  observation  and  communication  with 
their  staff  and  their  own  subordinate  commanders. 


Estimate  Support  Staff  Requirements 


I  This  principally  involves  setting  realistic  goals  based  upon  availability 

i  of  personnel  at  the  field  locations  where  the  training  system  is  to  be  used, 

|  analyzing  both  the  workload  imposed  by  the  system  and  the  military  knowledge 

[  requirements  for  role  playing  functions  and  successively  refining  the  system 

1  until  it  can  be  operated  by  the  available  personnel. 

A  preliminary  design  based  upon  current  training  systems  can  be  used  to 
detail  all  of  the  functions  to  be  portrayed.  These  functions  can  be  further 
described  in  terms  of  how  the  current  design  is  intended  to  operate  and  in 
i  terms  of  what  control  functions  will  be  required  for  the  current  design  for 

model  control,  role  playing,  and  evaluation. 
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Data  on  the  number  of  simulation  events  (e.g.,  movements,  fire  missions,  and 
air  strikes),  to  be  portrayed  per  unit  time  can  be  derived  from  current  command 
post  or  field  training  exercises.  Similarly,  data  on  the  number  of  voice  and 
digital  transmissions  per  unit  time  can  be  obtained.  Average  time  per  message 
can  be  directly  obtained  from  the  transmission  data  to  estimate  the  average  and 
peak  communications  workload  for  controllers. 

To  estimate  model  control  workload,  initial  values  can  be  obtained  analyti¬ 
cally,  but  for  greater  accuracy,  it  is  recommended  that  a  prototype  of  the 
interface  to  the  simulation  be  created  as  one  of  the  first  steps  in  system 
development.  Analysis  of  time  and  errors  in  the  use  of  the  interface  can  re¬ 
veal  potential  data  entry  bottlenecks  and  areas  of  model  representation  that 
demand  further  automation  before  the  simulation  software  is  built.  Performance 
tests  on  the  prototype  interface  should  use  inexperienced  personnel  to  insure 
that  excessive  controller  training  requirements  are  not  built  into  the  system. 

Workload  estimates  from  this  preliminary  analysis  can  be  used  to  determine 
the  minimum  number  of  controllers  required  to  perform  role  playing  functions 
and  the  minimum  number  of  additional  personnel  needed  to  meet  model  control 
requirements.  This  number  added  to  the  training  management  and  evaluation 
staff  is  the  minimum  required  by  the  preliminary  design.  To  reduce  the  abso¬ 
lute  number  of  people  required,  "doubling-up"  of  functions  should  be  examined, 
adding  model  control  tasks  to  role  play  and  evaluation  personnel  and  giving 
multiple  roles  to  role  players  to  the  extent  that  military  knowledge  require¬ 
ments  will  permit.  A  comparison  to  staffing  goals  will  then  probably  indicate 
that  further  reductions  are  needed.  The  data  from  the  prototype  interface  can 
be  used  to  target  the  most  time  consuming  functions  for  redesign. 

Final  Remarks 

We  recommend  two  radical  departures  from  incremental  improvement  of  exist¬ 
ing  systems.  These  are  (1)  base  the  simulation  model  upon  a  systemic  analyti¬ 
cal  wargame  and  (2)  begin  development  of  on-line  performance  measurement 
techniques  based  upon  the  interaction  of  the  training  audience  with  a  tactical 
data  system.  These  steps  are  believed  to  be  necessary  to  cope  with  the  most 
critical  shortcomings  of  current  command  staff  training  systems:  excessive 
control  personnel  overhead  and  the  lack  of  performance  measurement  capabili- 
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